Method and apparatus for executing voice command in electronic device

ABSTRACT

An apparatus and method for executing a voice command in an electronic device. In an exemplary embodiment, a voice signal is detected and speech thereof is recognized. When the recognized speech contains a wakeup command, a voice command mode is activated, and a signal containing at least a portion of the detected voice signal is transmitted to a server. The server generates a control signal or a result signal corresponding to the voice command, and transmits the same to the electronic device. The device receives and processes the control or result signal, and awakens. Thereby, voice commands are executed without the need for the user to physically touch the electronic device.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation of U.S. patent applicationSer. No. 15/433,196 filed on Feb. 15, 2017 which claims the benefit ofthe earlier U.S. patent application Ser. No. 13/903,345 filed on May 28,2013 and assigned U.S. Pat. No. 9,619,200 issued on Apr. 11, 2017 whichclaims the benefit under 35 U.S.C. § 119(a) to a Korean patentapplication filed in the Korean Intellectual Property Office on May 29,2012, and assigned Serial No. 10-2012-0057044, the entire disclosure ofwhich is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates generally to an electronic device. Moreparticularly, the present disclosure relates to an apparatus and amethod for executing a voice command in an electronic device.

BACKGROUND

In recent times, as multimedia technologies have grown, electronicdevices having multiple functions have proliferated. Examples of suchmulti-function devices include portable terminals such as smart phones,tablet PCs, smart cameras, as well as fixed home-based devices such aselectronic devices integrated with home kitchen appliances. Theelectronic devices mostly include a convergence function which combinesa number of functions.

Portable terminal designers strive to achieve advanced performance aswell as the convergence function, as well as slim and aesthetic designsof the device as a whole. Terminal manufacturers compete to presentsubstantially the same or advanced performance and to engineer newmodels that are smaller and slimmer than prior designs.

Among the various functions available, a recently commercialized deviceprovides a voice recognition function of relatively high accuracy. Sucha voice recognition function accurately recognizes a user's voice toeasily execute a corresponding function of the device without having topress a separate button or touch a key or touchscreen.

For example, the voice recognition function allows the user to make acall or write a text message without separate manipulation in a portableterminal such as a smart phone, to send the generated message, and toeasily set various functions such as a route planner, Internet search,and alarm.

To execute the voice recognition function, the related art drives acorresponding voice recognition application, activates the voicerecognition function, and then performs the corresponding function.

However, to perform the voice recognition, the voice recognitionapplication is initially started in response to a touch input command ona separate key or the touchscreen. This operation goes against theunique function of the voice recognition for facilitating data inputwithout touch. Further, launching the voice recognition applicationrequires finding it in a display screen including various applicationobjects, which may be difficult and time consuming in somecircumstances.

SUMMARY

Embodiments of apparatus and methods for executing a voice command in anelectronic device are disclosed. In an exemplary embodiment, a voicesignal is detected and speech thereof is recognized. When the recognizedspeech contains a wakeup command, a voice command mode is activated, anda signal containing at least a portion of the detected voice signal istransmitted to a server. The server generates a control signal or aresult signal corresponding to the voice command, and transmits the sameback to the electronic device. The device receives and processes thecontrol or result signal, and awakens. Thereby, voice commands areexecuted without the need for the user to physically touch theelectronic device.

In various embodiments:

The voice signal may comprise the wakeup command followed by the voicecommand.

The wakeup command may also comprise the voice command.

A silence duration may be determined between the wakeup command and thevoice command.

Processing the control signal or the result signal may compriseexecuting a particular application of the electronic device.

Processing of the control signal or the result signal may comprisedisplaying data corresponding to the result signal.

Once the voice command mode is activated, an object may be activated ona display indicative of the voice command mode being activated.

When a screen is locked prior to recognizing the wakeup command in thespeech, the screen may be unlocked responsive to the recognized wakeupcommand.

The speech may be recognized to contain a predetermined wakeup commandonly if a predetermined speaker of the voice signal is recognized. Thewakeup command may be detected automatically when the voice of thepredetermined speaker is recognized.

Alternatively, the wakeup command may be detected when the voice of thepredetermined speaker is recognized and a predetermined wakeup commandis recognized within the speech of the predetermined speaker. In anotherembodiment, a method for executing a voice command in an electronicdevice, comprises: detecting a voice signal which contains at least oneof a wakeup command and a voice command; transmitting the voice signalto a server; awakening the electronic device upon receiving a resultsignal indicative of the server detecting the wakeup command in thevoice signal; receiving a control signal or a result signalcorresponding to the voice command from the server; and processing thecontrol signal or the result signal corresponding to the voice command.

In an embodiment, a method operable in a server, for supporting a voicecommand of an electronic device, comprises: receiving a transmittedvoice signal which contains at least a voice command, from theelectronic device; generating a control signal or a result signalcorresponding to the voice command by recognizing and analyzing thevoice command; and sending the control signal or the result signalcorresponding to the first voice command, to the electronic device.

In an embodiment, an electronic device comprises: one or moreprocessors; a memory; and one or more programs stored in the memory andconfigured for execution by the one or more processors, wherein theprogram comprises instructions for detecting a voice signal andrecognizing speech thereof; when the speech is recognized to contain awakeup command, activating a voice command mode and transmitting atransmit signal containing at least a portion of the detected voicesignal to a server; and receiving and processing a control signal or aresult signal generated and transmitted by the server in response to avoice command within the transmit signal recognized by the server.

According to another aspect of the present invention,

Other aspects, advantages, and salient features of the invention willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses exemplary embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainexemplary embodiments of the present invention will be more apparentfrom the following description taken in conjunction with theaccompanying drawings, in which:

FIG. 1A is a block diagram of an electronic device for executing a voicecommand according to an exemplary embodiment of the present invention;

FIG. 1B is a diagram of a system for executing voice commands accordingto an embodiment;

FIG. 2 is a flowchart of a method for executing wakeup and voicecommands in an electronic device according to one exemplary embodimentof the present invention;

FIG. 3 is a flowchart of a method operable in a server according to oneexemplary embodiment of the present invention;

FIG. 4 is a flowchart of method for executing a voice command in anelectronic device according to another exemplary embodiment of thepresent invention;

FIG. 5 is a flowchart of another method operable in a server accordingto another exemplary embodiment of the present invention;

FIG. 6 is a flowchart of a method for executing a voice command in anelectronic device according to yet another exemplary embodiment of thepresent invention;

FIG. 7 illustrates a voice signal including a wakeup command and thevoice command that may be detected and recognized according toembodiments of the present invention;

FIG. 8A, FIG. 8B and FIG. 8C illustrate a dialing based on the voicesignal including the wakeup command and the voice command according toan exemplary embodiment of the present invention; and

FIG. 9A and FIG. 9B depict screenshots for illustrating a screenunlocked through wakeup command detection according to an exemplaryembodiment of the present invention.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components and structures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of exemplaryembodiments of the invention as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the embodiments described hereincan be made without departing from the scope and spirit of theinvention. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of theinvention. Accordingly, it should be apparent to those skilled in theart that the following description of exemplary embodiments of thepresent invention is provided for illustration purpose only and not forthe purpose of limiting the invention as defined by the appended claimsand their equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

By the term “substantially” it is meant that the recited characteristic,parameter, or value need not be achieved exactly, but that deviations orvariations, including for example, tolerances, measurement error,measurement accuracy limitations and other factors known to those ofskill in the art, may occur in amounts that do not preclude the effectthe characteristic was intended to provide.

Exemplary embodiments of the present invention provide an apparatus anda method for executing a voice command in an electronic device and aserver.

FIG. 1A depicts an electronic device 100 for executing a voice commandaccording to an exemplary embodiment of the present invention.Electronic device 100 can be any one of a variety of fixed or portabledevices. A portable device can be a portable terminal, mobile terminal,mobile pad, media player, tablet computer, smart phone, anotebook/desktop computer, a Personal Digital Assistant (PDA), a smartcamera, and so forth. The electronic device may be a portable electronicdevice combining two or more functions of those devices. An example of afixed electronic device is an electronic display device attached to ahome appliance such as a kitchen appliance.

Electronic device 100 can include a controller 110, a speaker/microphone112, a camera 120, a Global Positioning System (GPS) receiver 130, aRadio Frequency (RF) unit 140, a sensor module 150, a touch screen 160,a touch screen controller 165, and an external memory 170.

Briefly, according to embodiments of the invention, device 100 detects avoice signal and recognizes speech in the detected signal. When thespeech is recognized to contain a wakeup command, device 100 activates avoice command mode. In the voice command mode, device 100 is able torespond to subsequent voice commands. Thus, if the device is in a lockedscreen state or an idle state (or in both of these states if conditionsallow), the device can be awakened through the wakeup commandrecognition without the need for the user to physically touch a key ortouchscreen on the device 100. Once the wakeup command is detected, aspeech recognition process is performed to discern if a voice commandhas been uttered. If a voice command is recognized, the device 100 canthen perform an operation associated with that command.

In one embodiment, both the wakeup command detection and the voicecommand recognition are performed in the device 100. In anotherembodiment, the wakeup command detection is performed at device 100 andthe voice command recognition is performed at a server, following atransmission of a portion of the voice signal from the device 100 to theserver. In still another embodiment, both the wakeup command detectionand the voice command detection are performed at the server.

The controller 100 can include an interface 101, one or more processors102 and 103, and an internal memory 104. In some cases, the wholecontroller 110 may be referred to as the processor. The interface 101,the application processor 102, the communication processor 103, and theinternal memory 104 can be separate components or integrated onto one ormore integrated circuits.

The application processor 102 performs various functions for theelectronic device by running various software programs and thecommunication processor 103 processes and controls voice communicationand data communication. In addition to those typical functions, theprocessors 102 and 103 also execute a particular software module(instruction set) stored in the external memory 170 or the internalmemory 104 and conduct particular functions corresponding to the module.That is, the processors 102 and 103 carry out the method of the presentinvention in association with software modules stored in the externalmemory 170 or the internal memory 104.

According to one exemplary embodiment of the present invention(corresponding to the method of FIG. 2) the application processor 102receives a voice signal including a wakeup command and a subsequentvoice command from a user through the microphone 110, and performsspeech recognition on the voice signal to detect the presence of thewakeup command. When the wakeup command is detected, the applicationprocessor 102 may detect a silence duration between the wakeup commandand the subsequent voice command in the voice signal. Thus theapplication processor 102 determines whether the portion of the voicesignal corresponding to the voice command begins, and when it does, theapplication processor 102 sends that portion of the voice signal to aserver. Next, the application processor 102 receives a voice recognitionresult corresponding to the voice command from the server and performs acorresponding operation based on this result.

According to another exemplary embodiment of the present invention(corresponding to the method of FIG. 4), the application processor 102sends to the server all of the voice signal including the wakeup commandportion and the voice command portion, and receives a speakerverification result corresponding to the wakeup command from the server.When the result indicates that the wakeup command is detected, theapplication processor 102 activates the system. Next, the applicationprocessor 102 receives a voice recognition result corresponding to thevoice command and performs the operation based on the voice recognitionresult.

According to yet another exemplary embodiment of the present invention(corresponding to the method of FIG. 6), the application processor 102receives the voice signal including the wakeup command and the voicecommand from the user through the microphone 110, and performs thewakeup command detection using speech recognition. When the wakeupcommand is detected, the application processor 102 uses a voicerecognition algorithm to recognize a voice command in a subsequentportion of the voice signal and performs the corresponding operationbased on the recognized voice command.

One or more voice recognition processors and a speaker verificationprocessor can be a part of the application processor 102, or can beprovided as separate processors. The voice recognition processor and thespeaker verification processor may be unified, and include a pluralityof processors for different functions according to their implementation.The interface 101 interconnects the touch screen controller 165 of theelectronic device with the external or internal memory 170 or 104.

The sensor module 150 is coupled to the interface 101 to allow variousfunctions. For example, a motion sensor and an optical sensor can becoupled to the interface 101 to detect a motion of the electronic deviceor to detect the light from the outside. Besides these, other sensorssuch as position determining system, temperature sensor, or biometricsensor can be connected to the interface 101 to conduct relevantfunctions.

The camera 120 is coupled to the sensor module 150 through the interface101 to perform a camera function such as photo and video clip recording.

The RF unit 140, which may include at least one processor, performs acommunication function. For example, under control of the communicationprocessor 103, the RF unit 140 converts an RF signal to a basebandsignal and provides the baseband signal to the communication processor103, or converts a baseband signal output from the communicationprocessor 103 to an RF signal and transmits the RF signal through anantenna ANT. Here, the communication processor 103 processes thebaseband signal according to various communication schemes. For example,the communication scheme can include, but not limited to, a GlobalSystem for Mobile communication (GSM) communication scheme, an EnhancedData GSM Environment (EDGE) communication scheme, a Code DivisionMultiple Access (CDMA) communication scheme, a W-CDMA communicationscheme, a Long Term Evolution (LTE) communication scheme, an OrthogonalFrequency Division Multiple Access (OFDMA) communication scheme, aWireless Fidelity (Wi-Fi) communication scheme, a WiMax communicationscheme, and/or a Bluetooth communication scheme.

The speaker/microphone 110 can input and output an audio signal such asone for voice recognition (used during a training process to traindevice 100 to recognize a particular speaker and/or wakeup commandand/or voice command), voice reproduction, digital recording, andtelephone function. That is, the speaker/microphone 110 converts thevoice signal to an electric signal or converts the electric signal tothe voice signal. An attachable and detachable earphone, headphone, orheadset (not shown) can be connected to the electronic device through anexternal port.

The touch screen controller 165 can be coupled to the touch screen 160.The touch screen 160 and the touch screen controller 165 can detect thetouch and the motion or their cessation using, but not limited to,capacitive, resistive, infrared and surface sound wave techniques fordetermining one or more touch points with the touch screen 160 and amulti-touch detection technique including various proximity sensorarrays or other elements.

The touch screen 160 provides an input/output interface between theelectronic device and the user. That is, the touch screen 160 forwards auser's touch input to electronic device 100. The touch screen 160 alsopresents an output of device 100 to the user. That is, the touch screen160 presents a visual output to the user. Here, the visual output can berepresented as text, graphic, video, and a combination of these.

The touch screen 160 can employ various displays, examples of whichinclude, but are not limited to, Liquid Crystal Display (LCD), LightEmitting Diode (LED), Light emitting Polymer Display (LPD), Organic LED(OLED), Active Matrix OLED (AMOLED) or Flexible LED (FLED).

The GPS receiver 130 converts a signal received from an “artificial”satellite, to information such as location, speed, or time. For example,a distance between the satellite and the GPS receiver 130 can calculatedby multiplying the speed of light by a signal arrival time, and measuresthe location of the electronic device using the well-known triangulationby obtaining accurate positions and distances of three satellites.

The external memory 170 or the internal memory 104 can include fastrandom access memory such as one or more magnetic disc storage devicesand/or non-volatile memory, one or more optical storage devices, and/ora flash memory (e.g., NAND and NOR).

The external memory 170 or the internal memory 104 stores software.Software components include an operating system software module, acommunication software module, a graphic software module, a userinterface software module, an MPEG module, a camera software module, andone or more application software modules. Since the module being thesoftware component can be a set of instructions, the module can bereferred to as an instruction set. The module may be referred to as aprogram.

The operating system software includes various software components forcontrolling general system operations. The control of the general systemoperations includes, for example, memory management and control, storagehardware (device) control and management, and power control andmanagement. The operating system software may process normalcommunication between various hardware devices and the softwarecomponents (modules).

The communication software module allows communication with otherelectronic devices such as computer, server, and/or portable terminal,through the RF unit 140. The communication software module is configuredin a protocol architecture of the corresponding communication scheme.

The graphic software module includes various software components forproviding and displaying graphics on the touch screen 160. The term‘graphics’ embraces text, webpage, icon, digital image, video,animation, and the like.

The user interface software module includes various software componentsrelating to a user interface. The user interface software module isinvolved in the status change of the user interface and the condition ofthe user interface status change.

The camera software module includes camera related software componentsallowing camera related processes and functions. The application moduleincludes a browser, an e-mail, an instant message, a word processing,keyboard emulation, an address book, a touch list, a widget, DigitalRight Management (DRM), voice recognition, voice reproduction, aposition determining function, a location based service, and the like.The memories 170 and 104 can include an additional module (instructions)in addition to the above-stated modules. Alternatively, if necessary,some module (instructions) may not be used.

Herein, the application module includes an instruction for carrying outa speaker recognition function or a speech recognition function and avoice command execution function. The instructions according toexemplary embodiments of the present invention correspond to those forexecuting the operations illustrated in FIGS. 2, 4 and 6.

The various functions of electronic device 100 as mentioned above and tobe explained, can be executed in hardware and/or software and/or theircombination including one or more signal processing and/or ApplicationSpecific Integrated Circuits (ASICs).

FIG. 1B illustrates a system 195 for executing voice commands accordingto an embodiment of the present invention. System 195 includes theportable terminal 100 which communicates with a server 190 through anetwork 180. Server 190 can be e.g., a home network server, or a remoteserver accessed through a large network such as the Internet.Alternatively, server 190 can be a third party portable electronicdevice capable of performing a speech/voice/speaker recognition andanalysis function on voice signals transmitted thereto. Server 190includes a minimum of at least one processor 192 and a memory 194 toperform a host of operations. Exemplary operations of the server 190 inconjunction with electronic device 100 will be described hereafter.

FIG. 2 is a flowchart of a method 200 for executing wakeup and voicecommands in the electronic device 100 according to one exemplaryembodiment of the present invention.

At step 201, the electronic device 100 detects a voice signal which maycontain the wakeup command and the voice command from the user throughthe microphone 110. The wakeup command activates a voice command mode ofthe system, in which no touch contact is required with the touchscreenor a key in order to receive and analyze a voice command. Prior toreceiving this voice signal, the device 100 can be in an idle mode or alockscreen mode. In some embodiments, prior to receiving the voicesignal, the device 100 can be in an application execution mode in whichno listening for voice commands or operations responsive to voicecommands are executed.

In the following description, it will be assumed that the wakeup commandis typically independent of a voice command that temporally follows thewakeup command. However, in some “speaker-dependent” embodiments alsodiscussed below, any voice signal detected to be spoken by apredetermined speaker can serve as a wakeup command. In still otherembodiments (speaker dependent or speaker independent), the wakeupcommand also contains an inherent voice command. In the latter case, thewakeup command both activates the voice command mode and is a catalystfor the device 100 to perform an additional predetermined task, such asrunning a predefined application set by the user.

For example, the wakeup command can instruct to switch to a mode forinputting the voice command (“voice command mode”) and/or to unlock thescreen. The voice command executes various functions provided by theelectronic device 100. For example, the voice command executes dialing,photographing, MP3 playing, and so on. In various implementations, thevoice command can request the server 190 to search a map and to plan aroute.

In step 202, the electronic device 100 performs speech recognition onthe voice signal to discern whether the voice signal contains apredetermined wakeup command. This speech recognition can include aspeaker dependent recognition scheme in one embodiment, or a speakerindependent recognition scheme in another embodiment. Additional schemesare possible where a number of different wakeup commands arepredetermined, in which one or more predetermined commands is a speakerdependent wakeup command and one or more other commands is a speakerindependent command.

According to the speaker dependent recognition scheme, a particularspeaker or user needs to train a recognizer with his/her own voice inadvance. In this case, the speech recognizer can recognize only thespeech of the trained voice. The speaker independent recognition schemecan recognize speech of an arbitrary speaker voice. The speakerindependent recognition scheme extracts and databases information abouthundreds or thousands of voices in advance, and thus any user can use itwithout a separate training process.

Using the speaker dependent recognition, in some embodiments, thespeaker can be verified using the voice command portion of the voicesignal (which may comprise the entire voice signal). Hence, there is noneed to input a separate wakeup command. For example, when the speakeris verified using unique voice characteristics of the user, there is noneed to input the separate wakeup command. Accordingly, in theseembodiments, the voice command can also operate as the wakeup command.Thus, in steps 202 and 204, the specific voice of the particular user isrecognized, and wakeup command is automatically detected via thisrecognition.

Alternatively, when using the speaker dependent recognition scheme, aspeaker verification is performed based on the information of a voicesignal corresponding to a word(s) that is predefined, or set by a userin advance through repeated voice training. The user can train thedevice 100 to verify the speaker (and a specific wakeup command(s)) byinputting his voice corresponding to a predefined text. In so doing, itis necessary to input the wakeup command. Herein, the predefined textcan be input directly by the user or converted by inputting the voiceseveral times. The electronic device 100 or the server 190 can convertthe voice to the text.

When the wakeup command is detected in step 204, the method proceeds tostep 206. Otherwise, it returns to 201.

Although not illustrated in FIG. 2, when the wakeup command issuccessfully detected, an “object for recognizing the voice command” isactivated on the display in the locked screen (see FIG. 8A). Thisobject, which may be a virtual microphone, indicates activation of thevoice command mode, i.e., that the device is actively listening forvoice commands. At this time, the object may be firstly displayed, or,displayed in an emphasized manner if a faded version was previouslyvisible, in the locked screen. A Graphical User Interface (GUI) relatingto the speech/voice recognition can also be displayed at this time.Alternatively, when the wakeup command is detected, the displayed objectcan be activated and the GUI relating to the voice recognition can beimmediately displayed in the unlocked screen.

In an embodiment variation, when the wakeup command is detected in theidle mode and the screen is locked, the object for recognizing the voicecommand and the GUI relating to the voice recognition are displayedtogether. When the screen is not locked, the object for recognizing thevoice command and the GUI relating to the voice recognition can also bedisplayed together.

In step 206, the electronic device 100 detects a silence duration (ifone exists) between a first portion of the detected voice signal(hereafter, “first voice signal”) corresponding to the wakeup commandand a second portion of the detected voice signal (hereafter, “secondvoice signal”) corresponding to the voice command. Of course, thisassumes that the voice command is a separate entity than the wakeupcommand (as mentioned above, an embodiment is possible where the wakeupcommand is also a voice command). For example, assuming that the wakeupcommand is “Hi Galaxy” and the voice command is “Call Hong Gil-dong”,when the user consecutively pronounces “Hi Galaxy” and “Call HongGil-dong”, a silence duration exists between “Hi Galaxy” and “Call HongGil-dong”.

A short pause between two words in the detected speech can be used todetect the start of the voice command. In an embodiment, an extraneousportion of the detected voice signal immediately following the wakeupcommand can be blocked from being sent to the server together with theensuing voice command. For doing so, a Voice Activity Detection (VAD)technique can be used. For example, a voice signal typically has moreenergy than a background noise signal including the “silence” period.However, when background noise is low, unique characteristics of thehuman voice can be additionally identified. Typically, the uniquecharacteristics of the human voice are identified by observing energydistribution throughout various frequencies. The human voice includescharacteristics signature but no noise. Hence, the VAD technique candistinguish speech from a silence period including background noise.Accordingly, in an embodiment, instead of transmitting to the server anaudio signal including all sounds detected subsequent to the wakeupcommand, the device 100 waits until speech is detected, and thereaftertransmits only sound signals beginning with the detected speech thatfollows the wakeup command. That is, the method 200 avoids transmittingsignals containing just noise of a silent period following a wakeupcommand detection.

In step 208, device 100 determines whether the second voice signalcorresponding to the voice command begins. For example, device 100checks the start point of the voice signal corresponding to “Call HongGil-dong”. When the second voice signal begins, device 100 at step 210sends the voice signal corresponding to the voice command (e.g., “CallHong Gil-dong”) to the server. (The portion of the voice signaltransmitted to the server is variously referred to herein as “thetransmit signal.) (When the voice signal corresponding to the voicecommand does not begin at 208, the flow returns to 206.) Advantageously,by transmitting the voice command to the server, device 100 is freedfrom the processor intensive task of recognizing the speech of the voicecommand.

In step 212, device 100 receives the voice recognition resultcorresponding to the voice command from the server. For example, theserver analyzes the voice command “Call Hong Gil-dong”, and sends acontrol signal corresponding to “Call Hong Gil-dong” to device 100 orsends a search result of the route planning request or the map searchrequest.

In step 214, device 100 performs the corresponding operation based onthe voice recognition result corresponding to the voice command, ordisplays a result corresponding to the voice recognition. For example,when receiving the control signal corresponding to “Call Hong Gil-dong”from the server, device 100 searches a phonebook for a phone number ofHong Gil-dong and tries to connect the call with the searched phonenumber. In the case of the map/route request, device 100 displays thesearch result of the route planning request or the map search request.Thereafter, the process ends.

FIG. 3 is a flowchart illustrating a method, 300, performed by server190 according to one exemplary embodiment of the present invention. Thismethod may complement the operations of the above-described method 200operating in device 100. In this embodiment, the server receives thetransmit signal, i.e., the voice signal corresponding to the voicecommand (e.g., “Call Hong Gil-dong”) from the electronic device (e.g.,transmitted at step 210 of FIG. 2) in step 301.

Next, the server analyzes the voice signal corresponding to the voicecommand using a voice recognition algorithm (equivalently, “speechrecognition” algorithm) in step 302. That is, the server analyzes thevoice signal to recognize speech and discern a voice command from therecognized speech. The server then determines whether the resultcorresponding to the voice recognition is a control signal in step 304.If so, the server sends the control signal corresponding to the voicerecognition to device 100 in step 306. For example, after recognizing“Call Hong Gil-dong”, the server provides the corresponding controlsignal to device 100 to instruct device 100 to call Hong Gil-dong at anassociated phone number extracted from a phone book storage thereof.

When the result corresponding to the voice recognition is not thecontrol signal, the server provides the result corresponding to thevoice recognition to the electronic device in step 308. Alternatively,the server sends image content containing the search result of the routeplanning request or the map search request, whereby the device 100displays the content.

Accordingly, in the exemplary embodiments of methods 200 and 300, theelectronic device fulfills the wakeup command detection and the serverfulfills the voice recognition of the voice command. According toanother exemplary embodiment of the present invention, the servercarries out both of the wakeup command detection and the voicerecognition of the voice command.

FIG. 4 is a flowchart of another example method, 400, performed indevice 100 according to another exemplary embodiment of the presentinvention. Here, in an idle mode and/or locked screen mode, device 100receives a voice signal including the wakeup command followed shortlythereafter (or continuously thereafter) by the voice command from theuser through the microphone 110 in step 401. As described earlier, thewakeup command, when recognized, activates the system. For example, thewakeup command can instruct to switch to the mode for inputting thevoice command and/or to unlock the screen. The voice command commandsexecution of various functions provided by the electronic device 100.For example, the voice command executes dialing, photographing, MP3playing, and so on.

In step 402, device 100 sends the entire voice signal including thewakeup command and the voice command, to the server as the transmitsignal. Next, a voice verification result corresponding to the wakeupcommand is received from the server (step 404). That is, when the serverdetects that the transmit signal contains the wakeup command, it sendsthe voice recognition result that is received in step 404; otherwise,the server may not send any recognition signal back to device 100. Forexample, when device 100 receives the recognition result at step 404,this indicates that the wakeup command was detected, and device 100activates the system in step 406. The system activation unlocks thescreen or switches from the idle mode to an active mode or voice commandmode. (With the system activated and in voice command mode, device 100may subsequently detect new voice signals containing voice commands asin step 401 and repeat steps 402-404 and subsequent steps accordingly.)

Next, device 100 receives the voice recognition result corresponding tothe voice command in step 408, and performs the operation based on thevoice recognition result or displays the result corresponding to thevoice recognition in step 410. For example, when receiving the controlsignal corresponding to “Call Hong Gil-dong” from the server, device 100searches the phonebook for the phone number of Hong Gil-dong and triesto connect the call with the searched phone number. In the map/routeexample, device 100 displays the search result of the route planningrequest or the map search request. Thereafter, the process ends, anddevice 100 may receive new voice signals at step 401 and forward theseto the server for processing, whereby the server may continue to respondby sending control signals and/or results corresponding to thesubsequent voice commands. That is, steps 401 through 410 may berepeated with relevant operations only for the voice commands, but ofcourse not for the wakeup command since the device 100 has already beenawakened.

FIG. 5 is a flowchart of an exemplary method 500 performed by server 190according to another exemplary embodiment of the present invention. Thismethod may complement the operations of the above-described method 400operating in device 100.

At step 501, the server receives the voice signal including the wakeupcommand and the voice command from the electronic device 501 (i.e., thetransmit signal transmitted at step 402). In step 502, the serveranalyzes the voice signal corresponding to the wakeup command using thevoice recognition algorithm. That is, the server analyzes the voicesignal corresponding to the wakeup command and thus determines whetheror not the wakeup command is detected. This operation may be the same asthat of steps 202 and 204 in FIG. 2 performed by device 100 in thatembodiment. Note that a speaker-dependent and/or speaker independentrecognition operation may be performed, as in the embodiment of FIG. 2.(Both types of recognition schemes may be employed if multiplepredetermined wakeup commands are under consideration.)

In step 504, the server provides a speech verification result to theelectronic device. (Note that step 504 may be omitted in otherimplementations.)

When the wakeup command is detected in step 506 as a result of thespeech recognition processing, the server then analyzes the voice signalcorresponding to the voice command using the voice recognition algorithmin step 508. That is, the server recognizes the speech corresponding tothe voice command, and generates a response signal corresponding to anaction to be performed by device 100 for the particularly discernedvoice command. By contrast, when the wakeup command is not detected inthe voice signal, the flow returns to step 501. To this end, the servermay transmit a signal informing device 100 that no wakeup command hasbeen detected, whereby device 100 may continue to transmit to the servernewly detected voice signals at step 501. In various implementations,when the voice signal corresponding to the previous voice command isnormal, the server can request and receive only the first voice signalcorresponding to the wakeup command.

Although not illustrated, the server can detect a silence durationbetween the voice signal corresponding to the wakeup command and thevoice signal corresponding to the voice command, and thus distinguishthe wakeup command and the voice command.

In step 510, the server notifies the wakeup command result and the voicerecognition result (the response signal) to the electronic device. Forexample, the server determines whether the wakeup command is detected byanalyzing whether the speech contains the phrase “Hi Galaxy”, analyzesthe voice command “Call Hong Gil-dong”, and thus sends the controlsignal corresponding to “Call Hong Gil-dong” to the electronic device100.

Next, the server finishes this process, and may be configured to listenfor subsequent voice signal transmissions from device 100 as in step501. To this end, suitable signaling between device 100 and server 190can be designed to inform server 190 if device 100 has returned to anidle mode or a lock screen mode. If so, the server would treat asubsequently received voice signal as one that may contain the wakeupcommand. If not, the server would naturally just listen for a new voicecommand.

FIG. 6 is a flowchart depicting operations of an example method, 600,performed by the electronic device according to yet another exemplaryembodiment of the present invention. In this embodiment, the electronicdevice carries out both the wakeup command detection and the voicerecognition for the voice commands.

Steps 601, 602, 604, 606 and 608 may be the same as steps 201, 202, 204,206 and 208, respectively, of FIG. 2. The following description of steps601-608 reiterates some of the concepts described in connection withsteps 201-208.

At step 601, in the idle mode or in the locked screen, the electronicdevice 100 receives the voice signal including the wakeup command andthe voice command from the user through the microphone 110. The wakeupcommand activates the system. For example, the wakeup command caninstruct to switch to the mode for inputting the voice command or tounlock the screen. The voice command executes various functions providedby the electronic device 100. For example, the voice command executesdialing, photographing, MP3 playing, and so on.

In step 602, device 100 analyzes the voice signal using a speechrecognition algorithm (voice recognition algorithm) to determine whetherthe voice signal contains the wakeup command. As explained earlier, ifspeaker-dependent recognition is employed, this operation may involvemerely detecting that the voice matches a predetermined voice, or thatthe predetermined voice also contains particular speech matching apredetermined wakeup command(s). Alternatively, a speaker-independentrecognition scheme may be utilized. When the wakeup command isrecognized in step 604, the flow proceeds to step 606; otherwise, itreturns to step 600.

In step 606, device 100 detects the silence duration between the voicesignal portion corresponding to the wakeup command and the voice signalportion corresponding to the voice command. For example, provided thatthe wakeup command is “Hi Galaxy” and the voice command is “Call HongGil-dong”, when the user consecutively pronounces “Hi Galaxy” and “CallHong Gil-dong”, the silence duration lies between “Hi Galaxy” and “CallHong Gil-dong”.

In step 608, the electronic device 100 determines whether the voicesignal corresponding to the voice command begins. For example, theelectronic device 100 checks the start point of the voice signalcorresponding to “Call Hong Gil-dong” in step 608.

When the voice signal corresponding to the voice command begins in step608, the electronic device 100 analyzes the voice signal correspondingto the voice command using the voice recognition algorithm in step 610.

In step 612, the electronic device 100 performs the correspondingoperation based on the recognized voice command. For example, when therecognized voice command is “Call Hong Gil-dong”, the electronic device100 searches the phonebook for the phone number of Hong Gil-dong andtries to connect the call with the searched phone number. Thereafter,the process ends.

FIG. 7 depicts an example voice signal including a wakeup command and avoice command that may be analyzed in the embodiments described above.The illustrative voice signal input to device 100 may contain a wakeupcommand and a voice command in succession. That is, the voice signal mayhave a portion 700 corresponding to the wakeup command and a portion 720corresponding to the voice command, which are successively input to theelectronic device. A silence duration portion 720 lies between thewakeup portion 700 and the voice command portion 720.

FIGS. 8A, 8B and 8C are screenshots depicting a dialing operation usingthe voice signal including the wakeup command and the voice commandaccording to an exemplary embodiment of the present invention. As shownin FIG. 8A, an icon object 800 for recognizing the voice command isactivated according to the voice signal portion 700 corresponding to thewakeup command. The voice command (“Call Hong Gil-dong”) correspondingto the voice command portion 720 of the voice signal is recognized asshown in FIG. 8B, and then the operation is conducted according to thevoice command. For example, the phone number of Hong Gil-dong issearched in the phonebook and the call connection automaticallycommences with, the searched phone number as shown in FIG. 8C.

FIGS. 9A and 9B depict screenshots of a screen unlocked through speechrecognition and control operations according to an exemplary embodimentof the present invention. FIG. 9A depicts an example lock screen; FIG.9B shows an example unlocked screen. The process of unlocking the lockedscreen to generate the unlocked screen as illustrated in FIGS. 9A and 9Bcan be performed in any of the above-described methods of FIGS. 2, 4 and6 (e.g., steps 214, 406, 410 or 612).

In response to detection of the wakeup command portion 700 of the voicesignal matching a predetermined wakeup command or matching a particularuser's voice, the locked screen of FIG. 9A is switched to the unlockedscreen of FIG. 9B. Although not depicted, the corresponding operationcan be performed by recognizing the voice command portion 720corresponding to the voice command (“Call Hong Gil-dong”) following thevoice signal 700 corresponding to the wakeup command after the screen isunlocked.

In the exemplary embodiments of the present invention described above,the wakeup command and the voice command are separated. Alternatively,the voice signal corresponding to the voice command can be used for bothof the speaker verification and the voice command. Namely, the speakeris verified with the voice signal corresponding to the voice command.When the speaker verification is successful, the corresponding functionof the electronic device can be controlled or executed according to thevoice command.

The above-described methods according to the present disclosure can beimplemented in hardware or software alone or in combination.

For software, a computer-readable storage medium containing one or moreprograms (software modules) can be provided. One or more programs storedto the computer-readable storage medium are configured for execution ofone or more processors of the electronic device and/or the server. Oneor more programs include instructions making the electronic deviceand/or the server execute the methods according to the embodiments asdescribed in the claims and/or the specification of the presentdisclosure.

Such programs (software module, software) can be stored to a randomaccess memory, a non-volatile memory including a flash memory, a ReadOnly Memory (ROM), an Electrically Erasable Programmable ROM (EEPROM), amagnetic disc storage device, a Compact Disc ROM (CD-ROM), DigitalVersatile Discs (DVDs) or other optical storage devices, and a magneticcassette. Alternatively, the programs can be stored to a memorycombining part or all of those recording media. A plurality of memoriesmay be equipped.

The programs can be stored to an attachable storage device of theelectronic device and/or the server accessible via the communicationnetwork such as Internet, Intranet, Local Area Network (LAN), Wide LAN(WLAN), or Storage Area Network (SAN), or a communication network bycombining the networks. The storage device can access the electronicdevice and/or the server through an external port.

A separate storage device in the communication network can access theportable electronic device/server.

As set forth above, since the detected voice signal including the wakeupcommand portion and the voice command portion is processed, the user caneasily execute the voice command.

In addition, since the wakeup command detection is fulfilled before thevoice command is executed, the voice command can be carried out based onsecurity/personal information protection.

While the invention has been shown and described with reference tocertain exemplary embodiments thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims and their equivalents.

What is claimed is:
 1. An electronic device comprising: one or moremicrophones; a communication module; a display unit; and one or moreprocessors, wherein the one or more processors are configured to:receive a voice signal including wakeup command, a voice commandsubsequent to the wakeup command, and a duration of silence between thewakeup command and the voice command through the one or more microphoneswhile displaying a lock screen in a locked state of the display unit,recognize the wakeup command in the voice signal, in response torecognizing the wakeup command in the voice signal, change the lockscreen to a screen for a voice command function including a visualobject related to voice recognition while maintaining the locked stateof the display unit, the visual object being indicative of the voicecommand function being activated, transmit a signal including the voicecommand of the voice signal through the communication module to a serverfor conducting speech recognition on the voice command, display, throughthe display unit and after the wakeup command and before execution ofthe voice command, a text, which is received from the server, the textindicating that the server is conducting the speech recognition on thevoice command, and after displaying the text, perform an operationresponsive to a control signal, corresponding to the recognized voicecommand, received from the server.
 2. The electronic device of claim 1,wherein the wakeup command is pre-determined or is configured with atleast one word which a user inputs when a speaker-dependent recognitionscheme which is based on information on a voice of the user is used, andthe information is acquired from the voice signal, corresponding to thewakeup command, spoken plural times by the user in advance.
 3. Theelectronic device of claim 1, wherein the one or more processors areconfigured to transmit, to the server, the signal including both of thewakeup command and the voice command.
 4. The electronic device of claim1, wherein recognizing, changing, transmitting and performing are ableto be executed by the one or more processors without a user input afterreceiving the voice signal including the wakeup command and the voicecommand.
 5. An electronic device comprising: one or more microphones; acommunication module; a display unit; and one or more processors,wherein the one or more processors are configured to: receive a voicesignal including a wakeup command, a voice command subsequent to thewakeup command, and a duration of silence between the wakeup command andthe voice command through the one or more microphones while displaying alock screen in a locked state of the display unit recognize the wakeupcommand in the voice signal, in response to recognizing the wakeupcommand in the voice signal, change the locked state to an active stateof a system of the electronic device, in response to the changing to theactive state, control the display unit to display a screen for a voicecommand function including a visual object related to voice recognition,the visual object being indicative of the voice command function beingactivated, transmit a signal including the voice command of the voicesignal through the communication module to a server for conductingspeech recognition on the voice command, display, through the displayunit and after the wakeup command and before execution of the voicecommand, a text, which is received from the server, the text indicatingthat the server is conducting the speech recognition on the voicecommand, and after displaying the text, perform an operation responsiveto a control signal, corresponding to the recognized voice command,received from the server.
 6. The electronic device of claim 5, whereinthe wakeup command is pre-determined or is configured with at least oneword which a user inputs when a speaker-dependent recognition schemewhich is based on information on a voice of the user is used, and theinformation is acquired from the voice signal, corresponding to thewakeup command, spoken plural times by the user in advance.
 7. Theelectronic device of claim 5, wherein the one or more processors areconfigured to transmit, to the server, the signal including both of thewakeup command and the voice command.
 8. The electronic device of claim5, wherein recognizing, changing, transmitting and performing are ableto be executed by the one or more processors without a user input afterreceiving the voice signal including the wakeup command and the voicecommand.
 9. A computer program product comprising one or morenon-transitory computer readable storage media having a computerreadable program stored therein, wherein the computer readable program,when executed on an electronic device, causes the device to: receive avoice signal including a wakeup command and a voice command subsequentto the wakeup command from a user, where there is a duration of silencebetween the wakeup command and the voice command; recognize the wakeupcommand while a touch screen of the electronic device is locked; inresponse to recognizing the wakeup command unlocking the touch screen;transmit a signal including the voice command of the voice signal to aserver for conducting speech recognition on the voice command andobtaining a recognized voice command at the server; receive a controlsignal corresponding to the recognized voice command from the server;and perform an operation in response to the control signal.
 10. Thecomputer program product of claim 9, wherein perform the operationcomprises execute a particular application.
 11. The computer programproduct of claim 9, wherein perform the operation comprises display datain response to the voice command.
 12. The computer program product ofclaim 9, wherein unlock the touch screen further comprises display agraphical user interface (GUI) related to the voice command.
 13. Thecomputer program product of claim 9, wherein recognize the wakeupcommand further comprises recognizing a predetermined speaker of thevoice signal.
 14. The computer program product of claim 9, whereinunlock the touch screen further comprises unlocking the touch screen inresponse to recognizing a predetermined speaker of the voice signal. 15.The computer program product of claim 9, wherein unlocking the touchscreen comprises unlocking the touch screen in response to recognizingthe wakeup command and a predetermined speaker of the voice signal. 16.The computer program product of claim 9, wherein the voice signal isinput through a microphone of the electronic device in an idle mode, andwhen the wakeup command is detected in the idle mode, a voice commandmode is activated.